The HIPS-2 Software for Image Processing: Goals and Directions

HIPS-2 is a set of image processing modules which provides a powerful suite of tools for those interested in research, system development and teaching. In this paper we first review the design considerations used to develop HIPS-2 from its predecessor (HIPS'). Then, a number of further developments are outlined which are under consideration. These include the development of a graphical user interface, more general approaches to color from the standpoint of color reproduction and linear models of color, and extensions to the software and data format to deal with production-oriented tasks. 1. THE HIPS SOFTWARE HJPS* is an image processing package for the UNIX environment.78 It was originally developed at New York University to support a research project on low bandwidth coding of images of American Sign Language14, and was completed in 1983. In 1991, the software was completely rewritten in a design which involved a collaboration between Sharplmage Software in New York and The Turing Institute in Glasgow, resulting in the HIPS-2 software. The HIPS and HIPS-2 packages are now in use at over 200 sites worldwide in a wide variety of application areas including research in computer vision and robotics, visual perception, satellite imaging, medical imaging, mechanical engineering, oil exploration, etc. In this section the design of HIPS is reviewed briefly. In later sections the new features of HIPS-2 as well as possible future directions will be discussed. The design and implementation of HIPS was started some ten years ago when the most common UNIX environment consisted of a mid-size computer (such as a DEC Vax) and terminals. Thus, HIPS was written based on the old model of UNIX software. It consists of a large set of commands (currently numbering over 200) which perform different basic image processing tasks (enlarge, rotate, Fourier transform, edge enhance, etc.). These are used within a Unix shell session, taking advantage of the UNIX pipe facility (denoted in a command line with a 'I ' character). For example, the following command neg < image I enlarge I fourtr mulseq filter inv.fourtr framevar would be used to compute a negative of the data in file 'image', enlarge the image, transform to the Fourier domain, multiply the transform by the transfer function in file 'filter' , back-transform to the image domain, and compute statistics on the resulting filtered image. The pipe operator automatically treats the output of each command as the input of the subsequent command. Each command may be controlled by various switches (to specify amount of rotation, enlargement, etc.), and more elaborate image processing is carried out using sequences of commands with intermediate results stored in temporary files. Thus, an experienced user will use the commands in an interactive fashion while developing an image processing * HIPS stands for the Human Information Processing Laboratory's Image Processing System, as that was the laboratory at New York University in which it was originally developed. 382 / SP1E Vol. 1964 Applications of Artificial Intelligence 1993 0-8194-1 200-7/93/$4.00 Downloaded From: http://spiedigitallibrary.org/ on 06/29/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx algorithm, but then develop shell scripts (macro files of shell commands) for more production-oriented tasks. The pipeline concept remains a flexible one because of the image data structure that HIPS uses. Each image or image sequence is preceded by an image header. In HIPS, this is ASCII text in a relatively fixed format which specifies the size of the image (number of rows, columns and frames) and the format of individual pixels (bits, bytes, short integers, integers, floats, complex, etc.) as well as various documentary fields. (Note that HIPS works with multiple image sequences as well as single images, where it is up to the user to interpret the meaning of the sequence: animation in time, stereo pair, spectral bands, etc.) Thus, each command in the above pipeline first reads the header, and then allocates image buffers sufficient to handle the size of input image. If a command changes anything about the image (its size or pixel format), it changes the header accordingly. In .addition, the header structure contains a sequence history, which is simply a list of all the HIPS commands which have been applied to the sequence. Each command automatically updates this history, so that the user may examine a sequence header to find out what commands gave rise to it. The structure of HIPS which supported these facilities was quite simple. It consisted of a small low-level library and a large collection of commands. The low-level library handled the details in cornmon between different programs: memory management, image header I/O, image sequence history update, error handling, and little else. Each HIPS program was an independent entity, but the form of programs was similar: read the header, allocate buffers, then for each image in the sequence: read, process and write the image. A principal design goal of HIPS since it became a commercial product (in mid-1983) was to keep it standardized. In particular, the advantage of UNIX software is the ability to use the same set of sources on a wide variety of hardware from different manufacturers. As such, HIPS is written in standard Kernighan and Ritchie6 C with as few system calls as possible, and using only those system calls (and include files) which are likely to be standard across different versions of Unix. Thus, we have not upgraded to ANSI C (function prototypes and the rest) as this would limit the machine-independence. The HIPS software has been used on nearly every UNIX hardware platform at this point. The only hardware-dependent part of HIPS is the device support for scanning or digitizing images and for displaying single images or image sequence animations. Here, device independence is not entirely possible, but device support has been developed for a variety of framebuffers and window systems (such as SunView, X, XView, NextStep, etc.). 2. FROM HIPS TO HIPS-2 There have been several developments since HIPS was first implemented. First, we were involved in a proct to develop software for the simulation of models of visual processing. This led to the EVE software (for Early Vision Emulation) which was based heavily on the HIPS design, although its lowlevel library was enhanced to make typical programs easier to write, an enhancement which was further developed in HIPS-2. But, the primary event was the development of HIPS-2 itself. HIPS-2 grew out of a number of discussions (between Sharplmage Software and The Turing Institute) about the deficiencies we perceived in HIPS. The obvious deficiency (discussed further in Section 3) is the lack of an easy to use, graphical interface to HIPS' capabilities. But, as a prelude to the development of such a capability, we saw it as necessary to first provide access to HIPS' capabilities for the programmer. In other words, HIPS always had a command-line interface, but the code did not provide a set of image processing subroutines. This was obviously a glaring omission in the original design. Since the provision of a subroutine interface would involve a substantial amount of reprogramming, some time and care was taken to specify what other aspects of HIPS could be improved with the rewrite. This led to a specification for HIPS-2 which was programmed in early 1991. Here, we review a number of the more SPJE Vol. 1964 Applications of Artificial Intelligence 1993 / 383 Downloaded From: http://spiedigitallibrary.org/ on 06/29/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx important capabilities which appeared in HIPS-2. In the design and implementation we were careful to keep the software reasonably upward-compatible (although the command interface did change slightly), and to continue to maintain the strict machine-independent approach to the code. 2.1 Subroutine interface The primary motivation for the move to HIPS-2 was to provide a subroutine interface to the HIPS image processing capabilities. However, it was clear that such an interface would be of use to a variety of applications with different needs, including uses from programs which otherwise did not make use of HIPS itself (e.g. did not read images in HIPS format, did not manipulate image headers, etc.). Therefore, the subroutine interface operates at several levels of abstraction. At the lowest level no reference to HIPS-specific data is required; a routine is provided with image sizes and pointers to buffers as parameters. At higher levels of abstraction, a routine is provided instead with a memory-resident image header structure. This structure is similar in content to the image header structure used for HIPS image files with the addition of a pointer to the memory-resident image data. The header structure includes all of the other details required by the routine including image geometry and format. 2.2 Automatic format conversion The reprogramming of the low-level image header library also involved the provision of routines for image 110 as well as header 110. This allowed us to easily provide a new capability to the command-level HIPS-2 interface which is a kind of minimal object-oriented facility. The HIPS command-level interface was often frustrating to the user for the following reason. HIPS manages a number of different image formats (the raster formats such as byte, integer, float, complex, and so on, as well as other nonraster formats). Some HIPS. programs will change the image format (e.g. a Fourier transform of a floating point image results in a complex image). Most HIPS programs are only equipped to handle a subset of image formats. Thus, with HIPS it was up to the user to keep track of what format image was output by one program and to convert it to a form required by the next program if necessary (by inserting a program such as ftoc in a pipeline to convert from float to complex). In HIPS-2, this is no longer necessary. Each program has coded in it the list of image formats that it knows how to deal with directly. Header and image 110 routines are provided which read images and, if necessary, convert them automatically to the nearest available type handled by a given program. The conversion routines do their best to preserve information whenever possible (for example, preference would be given to a conversion from short integer to long integer over converting to byte), and notify the user in any case that such a conversion is taking place. This minor convenience has made HIPS-2 far more pleasant to use. 2.3 Standardized command-level interface With HIPS-2 a routine is provided for the standardized parsing of command arguments. This has made it possible to simplify the command-level interface to HIPS-2 and make it more consistent across programs. In addition, this has made it possible to provide a standard set of switches across the entire set of HIPS-2 programs. These include a standard way of requesting a short usage message (quicker and far more abbreviated than the online manual pages) and specifying how binary (1 bit per pixel) images are to be unpacked, how real images are to be converted to complex and how complex images are to be converted to real. As a command-line system, HIPS-2 still requires far more learning than other systems (menu-driven or graphical interfaces), but this consistency across programs lessens the load on the novice user a bit. 384 / SPIE Vol. 1964 Applications of Artificial Intelligence 1993 Downloaded From: http://spiedigitallibrary.org/ on 06/29/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx